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ABSTRACT 

Properties of objects and spatial relations between objects play an important role in rule- 
based approaches for high-level vision. The partial presence or absence of such properties and 
relationships can supply both positive and negative evidence for region labeling hypotheses. 
Similarly, fuzzy labeling of a region can generate new hypotheses pertaining to the properties of 
the region, its relation to the neighboring regions, and finally, the labels of the neighboring 
regions. In this paper, we present a unified methodology to characterize properties and spatial 
relationships of object regions in a digital image. The proposed methods can be used to arrive at 
more meaningful decisions about the contents of the scene. 

1. Introduction 

The determination of properties of image regions and spatial relationships among regions 
is critical for higher level vision processes involved in tasks such as autonomous navigation, 
medical image analysis, or more generally, scene interpretation. In a rule-based system to 
interpret outdoor scenes, typical rules may be 

IF a REGION is THIN AND SOMEWHAT NARROW 

THEN it is a ROAD 

IF a REGION is RATHER BLUE AND HOMOGENEOUS AND 
IF THE REGION is ABOVE a TREE REGION 

THEN it is SKY 

Although humans may have an intuitive understanding of words such as "thin" and "narrow", 
such concepts defy precise definitions, and they are best modeled by fuzzy sets. Similarly, 
humans are able to quickly ascertain the spatial relationship between two objects, for example "B 
is above A", but this has turned out to be a rather elusive task for automation. When the objects in 
a scene are represented by crisp sets, the all-or-nothing definition of the subsets actually adds to 
the problem of generating such relational descriptions. It is our belief that definitions of 
properties and spatial relationships based on fuzzy set theory, coupled with a fuzzy segmentation 
will yield realistic results. 

Rosenfeld[l-3] defined many terms used in the analysis of spatial properties of objects 
represented by fuzzy sets. Pal has defined similar geometric attributes (such as index of area 
coverage) and have developed low- and intermediate-level algorithms based on such attributes 
[4], Dubois and Jaulent[5] generalized Rosenfeld’s definitions using both fuzzy set and evidence 
theories. 
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Approximate spatial relation analysis has also attracted the attention of many researchers 
in the past several years. In many situations, precise description of relations among objects may 
be too complex and computationally too expensive. Approximate spatial relation analysis 
provides a natural way to solve real world problems with a reasonable cost. Freeman[6] was 
among the first to recognize that the nature of spatial relations among objects requires that they be 
described in an approximate(fuzzy) framework. Retz[7] has examined the intrinsic, deictic, and 
extrinsic use of spatial prepositions and has designed a system called CITYTOUR that answers 
natural language questions about spatial relations between objects in a scene and about the 
movement of objects. More recently, Dutta[8] has applied fuzzy inference and used a 
generalization of Warshall's algorithm to reason about object spatial positions and motion. 
However, modeling spatial relations among image objects is not addressed in any of these papers. 
Keller and Sztandera[9] addressed the problem of defining some spatial relationships between 
fuzzy subsets of an image by using dominance relations of projections of the regions onto 
coordinate axes. 

In this paper, we propose direct methods to analyze properties of fuzzy image regions and 
spatial relations between fuzzy image regions quantitatively. The methods use membership 
functions generated by a fuzzy segmentation algorithm such as the fuzzy C-means algorithm [10]. 
The partition generated by the segmentation process is assumed to define C fuzzy subsets, one 
representing each object or region in the image. We express the membership function of each 

object in terms of its a-cut level sets and perform all computations on the level sets to obtain 
spatial properties of objects. We determine the relative positions of the level sets based on certain 
measurements on the elements of the level sets, and then we map the aggregated angle 
measurements into the interval [0,1] using suitable membership functions to define spatial 

relations between regions as fuzzy sets over the domain of a-levels. 

In section 2, we describe methods to generate fuzzy subsets that describe the objects 
(regions) in the image. In section 3, we review the existing methods to compute geometric 
properties and attributes of fuzzy image regions, and suggest how these methods can be easily 
extended to nongeometric properties and attributes. In section 4, we describe our method to 
compute membership functions for spatial relations between fuzzy regions. The relations include 
LEFT OF, RIGHT OF, ABOVE, BELOW, BEHIND, IN FRONT OF, NEAR, FAR, INSIDE, 
OUTSIDE, and SURROUND. In section 5, we show some typical experimental results of 
attribute and spatial relation analysis involving fuzzy image regions. Section 6 contains the 
summary and conclusions. 

2. Generation of Fuzzy Subsets to Describe Objects in the Image 

Prewitt [11] was the first to suggest that the results of segmentation be fuzzy subsets of the 
image. In a fuzzy representation of an image, each object is represented by a fuzzy region F, 

where F is defined over a referential set Q. Here, Q is the domain over which the image function 

is defined. In this paper, we are mainly concerned with the discrete case, and hence Q may be 

considered as a two-dimensional array. The membership function hf for the object is defined by: 

— > [0, 1]. Each point x = ( x,y ) in Q is assigned a membership grade Hf{x). It is further 

convenient to represent this region in terms of a-cut level sets F a as: F a = {x I fipix ) > a), where 

a e [0,1]. In a real image, the number of membership values present is finite, and can be made 

quite small by quantizing the values. Hence, they can be enumerated as l=aj>a 2 > . . . >a„. In 

what follows, a n +i will be assumed to be 0. The level sets are nested, i.e., F a &F a j for a/ < aj. 
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In addition, for each a-cut level set F a i, we can associate a basic probability assignment m(F a i), 
where m^F^i) satisfies: Z m^F^i) =1 [5]. 

One popular method for assigning multi-class membership values to pixels, for either 
segmentation or other types of processing, is the fuzzy C-means(FCM) algorithm[10]. The 
normalization of the memberships across classes in that approach sometimes leads to counter- 
intuitive memberships. The partition generated by FCM may also be sensitive to noisy features 
and outliers. Also, the number of classes must be specified for the algorithm to run. The 
possibilistic C Means algorithm and the unsupervised clustering algorithms proposed by the 
authors overcome many of these problems [12-13]. 

3. Properties and Attributes of Fuzzy Regions 

There are many ways to describe properties and attributes of an object. Properties and 
attributes of fuzzy image regions may be both geometric and non-geometric. In practical 
applications, some of the geometric properties that are frequently encountered are area, height, 
extrinsic diameter, intrinsic diameter, roundness, elongatedness, etc. [3]. Examples of non- 
geometric properties are brightness, color and texture. We now briefly summarize some geometric 
properties and their definitions from the existing literature [3]. 


The area of a fuzzy region F is defined as the scalar cardinality of F, i.e., 

a(F)- XM*) (1) 

jceft 

The height h of a fuzzy region F along the direction u is defined as 

h u (F) - ^max /t/r(u,v) (2) 

u 

where v is the direction perpendicular to u. Rosenfield[2] defined the extrinsic diameter of a fuzzy 
region F as 

E(F) = max h u {F) (3) 

where h u is defined as above. The geometric property "elongatedness" may be defined in terms of 
the ratio of the minor extrinsic diameter and the major extrinsic diameter, i.e., 
max h u (F) 

MEL(F) = 1- “ffip (4) 

Conversely, the geometric property "roundness" may be defined as the complement of 
"elongatedness". 

The geometric properties of objects can also be defined with respect to a-cut level sets [5]. 

Assume we have nested a-cut level sets {F°l c F a i c . . . c F a n}, with a basic probability 
assignment m defined by 

m(F a ‘) = a , . a,+i, (5) 


where a\ =1, and a n + 1 = 0. Then, for any x e F a i - F a >- 1, V-f{x) = The expected value of a 

property P(F), may be measured as: 
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P(F) is the expected value of P{F). Since F 0 ^ is a crisp set, traditional techniques can be used to 
compute P(F*Xi). For example, one may simple average the value of the property in the crisp 
region denoted by F 0 ' to obtain P(F a i). Dubois and Jaulent proved [5] that a(F) is the expected 

area a (F) and the height of F along the y-axis is equal to the expected height along the y-axis of 
F. For the expected extrinsic diameter, the following inequality is true, 

e(F)>E(F). (7) 

4. Spatial Relations between Fuzzy Regions 

The primitive spatial relations between two objects are[6]: 1) LEFT OF, 2) RIGHT OF, 3) 
ABOVE, 4) BELOW, 5) BEHIND, 6) IN FRONT OF, 7) NEAR, 8) FAR, 9) INSIDE, 10) 
OUTSIDE, and 11) SURROUND. In he following, we present detailed definitions and methods 
for computation of memberships for some of the relations. 

We define the relations as fuzzy sets over the universe of discourse of the a-cut values 

{ai, . . . ,oc n }. The general approach we use is as follows. Let A and B be two fuzzy sets defined 

on F2. At each a-cut value a j5 we compute the membership value for "A a i RELATION B a i" 

based on certain measurements yon the relative positions of the pairs of elements ( a,b ), a&A a i 

and b e B a i. These measurements are aggregated for all pairs elements to give an aggregated 

measurement y. The membership value for " A a i RELATION B a " denoted by (1arel_b («;) is 

then computed by evaluating a membership function //rel at y,. We are currently investigating 
methods to compute the overall membership for "A RELATION B", once the memberships for 

"A a i RELATION ZA" is computed for all a,. This may be achieved via a fuzzy aggregation 
operator, or from a method suggested by Dubois and Jaulent [5]. Ternary relations (such as "A IS 
BETWEEN B and C") can also be handled in a similar fashion. 

In the following, we discuss the membership functions Hrel for some of the relations 
listed above in more detail. In Section 5, we show examples of membership computations for a 
variety of relations in different situations. 

4.1 LEFT OF 

Human perception of spatial positions between two objects is closely related to angular 
information. For example, one would search a sector area subtending an angle of approximately 

180° left of oneself to find an object that supposedly lies to one’s left. Here, the distance between 
the person and the object is relatively unimportant. Based on this observation, we define most of 
the spatial relations in terms of angular measurements. 

Suppose we have two points A and B. Denote AB as the line connecting A and B. Let 6 be 
the angle between AB and the horizontal line, as shown in Figure 1. The membership function for 
"A is to the LEFT of B " may be defined as a function of 6 as 
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I0| < an/1 
an/1 <\6\<n/l 
I0| > n/l 


( 8 ) 


MLEFT (fi)= i 


1 

7t/2(l-a) 

0 




Figure 1: (a) Point relationship for "LEFT OF", (b) the membership function for "LEFT OF". 


A large value for a tends to give an optimistic result, and a small value would give a pessimistic 

result. Other symmetric functions may also be used to define /iLEFT- The definition in (8) assumes 
that A and B are points. If they are two fuzzy regions, the angles described above are computed 

and averaged for all pairs of elements ( a,b ), ae A*/ and b e B a <. The membership grade for "A a ‘ 

LEFT OF B a i" is obtained by mapping the averaged angle 6 0 through the membership function 
defined in (8). 

4.2 RIGHT OF, ABOVE, BELOW, BEHIND, IN FRONT OF 

These relations may be calculated similar to the relation "LEFT OF", using aggregated 
values of angles made by lines joining pairs of points along with a corresponding trapezoidal 
membership function. Due to the symmetry in our definitions, the membership grade for "A is to 
the LEFT OF B” is the same as that for "B is to the RIGHT of A". The symmetric property also 
applies to the relation pairs "ABOVE" - "BELOW", and "BEHIND" - "IN FRONT OF". It is to be 
noted that some of the terms mentioned above actually contain three dimensional information. As 
images are usually represented in a 2D space, some of these terms may not have any meaning. 

4.3 INSIDE, OUTSIDE 


For two level sets A a i and B a ‘, the membership function for the spatial relation "A a i is 
INSIDE B a i" may defined as. 
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WNSIDE(ai)= 


(9) 


\AainB a i\ 

\B a '\ 

where [A a ir\B a i\ and \B a i\ are the cardinalities of the level sets A a ‘ and B a ‘ respectively. In a 
digital image, cardinality of a set is the number of pixels that belong to the corresponding level 
set. The membership function for "A is OUTSIDE B" can be defined as the complement of that 
for "A is INSIDE B". 

4.4 SURROUND 


If we assume that all the level sets of an object are connected regions, at each a-cut level 
set, we can find two lines l\ and h for each point in 5*/, as shown in Figure 2. Let 0 denote the 
angle between the two lines as shown. The membership grade for "A a i SURROUNDS B a i" may 
be calculated by first computing the average 0, of the angles 0 for every element of #*», and then 
applying the following mapping at 0 = 0,. 

1 0 > (2 -a)n 

n< 0 < (l-a)n (10) 


HSURROUND(O)- 


n- 9 
?r(l-a) 


0 


d<n 



Consider three points A, B and C as shown in Figure 3. The membership value for "C is 
BETWEEN A and B" may be defined using a trapezoidal shape as shown in Figure 3. 

f 1 l0-7d < an/2 


M BETWEEN = ) 


n/1 - 1 n- fl| 
{l-a)Jt/2 
0 


an/1 < 10-nj < n/1 
10-nj > n/1 


(ID 
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The membership value for "C 0 * is BETWEEN A a i and B a i" may be computed by evaluating the 
membership function in (1 1) at 6 = <9„ where 0, is the average of all the angles between lines (a,c) 


and ( c,b ), where aeA a i and b e B a i and ce C a L Other spatial relations among objects may be 
defined in a similar way. 
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Figure 3: (a)Definition of the angle 0to compute the relation "BETWEEN", (b) the membership 

function for "BETWEEN". 


5. Examples of Spatial Relation Analysis 

Extensive simulations were conducted before we applied the proposed methods to real 
images. In the simulations, we chose objects with various membership function distributions, such 
as Gaussian shapes, triangular shapes, and exponential shapes. Relative positions and sizes of 
objects were also altered to observe the influence on the resulting membership functions for 
spatial relations. We first present two typical examples from our simulation experiments. We then 
present examples involving real images. 
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Figure 4(a) shows the fuzzy membership functions of two objects in an image and Figure 
4(b) shows the fuzzy membership functions of three objects. The z - axis represents the 
membership grades for the objects. In Figure 4(a), the large object lies below the small object. In 
Figure 4(b), the small object lies in between the two large ones. It is to be noted that the 
membership functions for the large objects are not symmetric about the peak value. The 
membership grades of two spatial relations in the two images are shown in Figure 5. From Figure 

5(a), we notice that at small a-cut levels, object A (large one) lies somewhat to the right of B. 
However, it is definitely below B. Therefore we have a reasonably high membership grade of 

about 0.85 for small a-cut levels for the relation "A is BELOW B". As the a-cut level increases, 
object A shrinks more and more to a position perfectly below object B. This results in a gradual 
increase of the membership grades to one. Similarly, in Figure 5(b), we initially have a low 

membership grade for A is BETWEEN B and C and as the a-cut level rises, object A's position is 
more BETWEEN B and C. Therefore the membership grades related to the spatial relation also 
increases accordingly. 



Figure 5: (a) Membership grades for "A is BELOW B" for the objects in Figure 4(a), (b) 
membership grades for "A is BETWEEN B and C " for the objects in Figure 4(b). 


We next present some typical examples of our experimental results with real images. 

Figure 6 shows a 256x256 image of a natural scene as well as its segmentation by the Gustafson- 
Kessel algorithm [12]. (The closest crisp partition is shown.) A texture feature (homogeneity) and 
three color features (red, green, and blue) were used to perform the segmentation. The segmented 
image shows three main objects: sky, road, and trees. Figures 7(a) and 7(b) show the membership 
grades for the "correct" spatial relation "The sky is ABOVE the trees" and the "false" spatial 
relation "The sky is to the LEFT of the trees". In the image, a considerable portion of the sky is 
actually lower than the tree region. However, our method still generated high membership grades 
for the true hypothesis. This shows that our method of aggregating relations is very effective in 
capturing the intuitively correct overall spatial relation between regions. The membership grades 
for "The sky is to the LEFT of the trees" is low, as expected. Figure 7(e) shows the plot of the 
membership function for the ternary relation "The Trees are BETWEEN the SKY and the 
ROAD", for the segmentation shown in Figure 6(b). As expected, our method generated high 
membership grades for this correct hypothesis. 
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6. Conclusions 

In this paper, introduce a new approach to analyze spatial relations between objects and 
among objects. In this approach, objects in the image are viewed as fuzzy regions, and spatial 
relations between fuzzy regions are viewed as membership functions (possibility distributions) 

defined over the set of a-cut sets of the fuzzy regions. This a-cut approach is similar to the 
approach introduced by Dubois and Jaulent ; and hence is consistent with the existing definition 
for the geometric properties of spatial regions. Since the properties and spatial relations are 

defined over the set of a-cut sets, efficient algorithms to compute these relations can be devised, 
and these algorithms save considerable computation time. The methodology expressed in the 
paper can be widely used in such areas as image understanding, rule-based reasoning, and motion 
analysis. 
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